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This docusent exaaines the educational output of 
three alternative approaches to introductory aacroeconoiics at the 
University of Notre Dase. The fraaevork for evaluation consists of 
the cognitive and affective tradeoffs entailed by using a new 
experimental course as opposed to two acre traditional eottr-<>s. The 
experiaental course is a freshnan level course which eaphasi2.vis 
laboratory and coaputer prograaing to teach hypothesis testing and 
econoaics aethodology. The other two courses^ lecture and discussion » 
are aore traditional in scope f although one of the courses emphasizes 
current econoaic probleas* Besults of the scores on the Test of 
Understanding College Econoaics indicate that students in the 
experiaental courses had higher scores than one of the traditional 
courses. In the skills of application of econoaic principles, the 
experiaental course students score better then the tvo traditional 
courses, while scoring the saae or less on recognition and 
understanding of econoaics principles. The affective goals are 
aeasured by course evaluations, change and final attitudes toward 
econoaics as a discipline, and changes in opinions toward econoaic 
issues. Results of the study indicate that the experiaental course 
had the highest student rating in spite of the heaviest work load. 
Students found the experiaental course aore useful and relevant than 
the other tvo. The experiaental course ranked second in its ability 
to change student attitudes about econoaic issues. (DE) 
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Introduction 

Many acononlcs departments offer several alternative introductory courses 
to the students of their colleges and universities. Eresunably this is the 
result of the need to respond to a conflaination of institutional req^irements as 
well as desired educational objectives. The rational department, of course, 
establishes a pattern of offerings which maximizes some objective function given 
institutional requirements and resource constraints.^ 

Such a maximization requires that the objectives of each alternative 
introductory course be carefully specified and measured. Besides providing 
information regarding the actual outcome of each course, this process allows 
a department to compare the alternative courses in terms of the tradeoffs 
between them and to determine whether these tradeoffs are acceptable. Unfortu- 
nately the methodology for such an evaluation is not well defined nor is the 
specification or measurement of objectives generally or easily available. 

This paper addresses these problems by providing evidence on the educational 
output of three alternative approaches to Introductory macroeconomics with a 
special focus on a new approach which has been developed at Notre Dame. It 
attenspts to measure the broad educational effects, both cogniblve and affective, 
of the courses. The fl*amework for evaluation will be the tradeoffs entailed 
by using the new course as opposed to two courses of a more familiar nature. 
Hopefully our attempt in this difficult area will encourage further efforts 



^e objective functions may differ in terms of what is maximized (or 
minimized). For example a department may choose from among the following 
alternatives: laaxlmize the number of students who take introductory economics, 
maximize the economic knowledge of students who take the introductory courses, 
maximize interest in economics as a major, maj:lffilze the usefulness of introductory 
economics with respect tc either the general educational background or vocational 
orientation of the student, and so on. Bach one of these seems to imply a 
different pattern of offerings or at least a variation in the content of a 
given pattern. 
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leading to laiprovenents in both the neasurement of educational output and the 
fliethodology of evaluation. 

The first section of the paper provides a brief discussion of the three 
courses, including a specification of their desired or expected outcomes as 
well as a siinple profile of the students enrolled in each course. The next 
two sections present partial analysis of results and tradeoffs. The first of 
the two examines cognitive achievement of students as measured by the Test of 
understanding College Economics (TUCE). The second section presents the avail- 
able evidence concerning affective outputs, e.g. student attitudes toward the 
respective courses; towards economics as a course of study; and toward certain 
economic problems > institutions, and poliCiei. The final section employs 
regression analysis to provide a "total analysis" of the output of the various 
courses. 

Description of the Alternative Approaches 

Each of the courses is an Introductory macroeconomlc course and thus they 
share certain comnon elements: 

(i) Treatment of the nature of certain macro problems and institutions; 
(ii) Treatment of certain tools of macro analysis; and 
(iii) Treatment of the application of the tools to the problems. 
Beyond these shared elements there are significant differences in the 

courses. Of mjor interest is Economics 101-102, the new approach which 

2 

teaches introductory economics as a "lab" course. 



^A more extensive description of this sequence as originally offered 
can be found in Economic Education Experiences of Enterprising Teachers , vol. 11, 
(New York: Joint Council on Economic Education, i^l^h PP« 95-101. 
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The course is offered to groups of approxinately thirty-five self-selected 
frestaMQ and it fulfills a university coci«a science requirement. The students 
are introduced to economics as a science and learn both economic theory and the 
technical skills of testing hypothesis. There are three meetings each week, 
tw lectures and a "lab"— a two-hour session vrtiich is conducted in a lab 
equipped with coaputer timesharing facilities. The first semester lectures 
cover two topics. The first is philosophy of science and here the students are 
introduced to the methodology of modern science with emphasis upon the process 
of analysis. The hypothesis testing methodology of economics is introduced 
within this broader scientific framework. The second topic is the content of 
macroeconomics. A typical model allowing for monetary and fiscal influences 
is constructed and then expanded to include aggregate supply. Policy implica- 
tions are also highlighted. In this discussion the theoretical antecedants 
of the postulated relationships and the empirical evidence bearing on their 
validity are explored as well. 

In the lab the students initially learn the basic tools of empirical 
research. They first learn to program in Fortran IV and attain the skiUs 
necessary to construct and manipulate large data sets. Next they learn the 
basic statistical concepts which are used in economic research. The final 
behavioral objective of this portion of the lab is the trograaming and use 
of a sin>le linear regression routine. The concluding lab sessions allow 
students to select certain macroecononic hypothesis drawn from the lecture 
and to test and evaluate these hypotheses for the U.S. econooy* 
determinants of consumption and of investment demand. These final sessions 
integrate lecture and lab, theory and methodology, by having the students perform 
siiVle research tasks. ^ The materials used in the course include two textbooks, 



3The second semester of the aeq^ience deals with microeconomics. Here the 
entire laboratory time is devoted to testing micro relationships about education, 
SgrItioS,lLS regional inequality. Thus the integration of lecture and lab is 
much iwre complete during the second semester. 
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one of wtilch is on philosophy of science, and a lab manual. 

The other two courses are of a more familiar variety. Economies 223 is a 
sophomore level course required of all Business College students. Two of the 
three meetings per week are lectures with approximately 300 students, the other 
is a small discussion class of approximately 20 students directed by a teaching 
assistant. The same basic macro model Is specified with some micro, government 
activity within a market economy, and poverty treated as additional topics. 
There is one text, a standard introductory book.'^ 

The third alternative, Economics 121, is another freshman course which 
fulfills the university social science requirement. Its 75 self-selected 
students meet three times each week, two sessions are lectures and the third 
may be either lecture, discussion, or problem session. While the basic macro 
model with Its policy dimensions are specified, the major concentration of the 
course Is on current policy Issues In macro, e.g. one-third of the course 
was spent on wage-price controls during Fall, 1973. In keeping with this general 
emphasis, three paperbacks were used, one of which was a short macroeconomlcB 
text. 



At Notre Dame the Bconomlcs Department Is In the College of Arts and 
Letters. 

^Although this sequence may be considered conventional. It too has been 
subject to change. In particular, during the time covered by our Investigation, 
Fall 1973, under a grant from the Alfred P. Sloan Foundation an experiment 
Involving a specially designed system of conputer assisted Instruction was 
conducted. In the escperlment certain students were allowed to use the eooputer 
BAterials In place of the regularly scheduled discussion classes. 
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The above dlacusslon highlights some of tho differences between the 
alternative approaches but does not explain why these differences exist. The 
structure for Economics 223 results from Its being the first of a two course 
sequence required of all College of Business students. As such it places a 
heavy emphasis on conventional macroeconomic content. The 121 course exists 
because it allows students of all. colleges to use economics to satisfy a social 
science requirement. It utilizes a current macro problems approach in order to 
stimulate student Interest in economics. The 101-102 sequence was created 
because we felt it was a "better way" to teach economics and at the same time 
better served the educational objectives of the freshman year. With respect 
to the former the 101-102 students would become familiar with economic content 
as well as the type of activities performed by professional economists.. In 
terms of the latter the sequence equips students with knowledge and skills 
which are applicable over a range of academic areas. 

In attempting to evaluate and thus compare the courses » our concentration 
is on the educational side. In particular, we are asking "if the economics 
department were to teach all Introductory courses in the form of 101 » what 
cognitive and affective tradeoffs would be involved?*'^ Given our specification 
of the respective educational objectives our expectations, are that there would 
be^ a loss in the pure content area as measured by TUGE when compared with 
the 223 approach. We would expect TUGE performance to be comparable to the 
national norm and to exceed that of 121. Turning to the affective domalnt 
the impact of 101 should be less than that of the more topical course 121* 



Note that once the educational side has been determined questions regarding 
institutional requirements and resource constraints would then be addressed. 
For instance the educational tradeoffs may be acceptable to us as a department 
but not to the College of Business or the hardware facilities necessary to 
process 600 students through 101-102 may preclude its adoption as the norm 
for the introductory course* 

ERIC 
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However we would expect that both 103. and 121 would be 8^pe^ior to 223 in this 
domain. The following sections will indicate that our expectations are 
generally correct and that we are able to speciiy the types of tradeoffs that 
would occur if 101 became the norm for introductory courses. 

Before tuiiiing to the results of the analysis of the tradeoffs among the 
courses it will be useful to indicate the characteristics of the students in 
each of the courses since no attempt was made to assign students randomly. 



TABLE I 

Selected Characteristics of Students 
(standard deviations in parenthesis) 



Characteristic 


101 


121 


223 


Verbal Scholastic Aptitude Test Score (VSAT) 


539.^ 
(76.3U) 


526.35 
(66.08) 


567.51 
(86.5if) 


Mathematical Scholastic i^titude Test Score (MSAT) 


602.78 
(69.27) 


590.00 
(68.98) 


620.09 
(78.0«f) 


Projected University Grade Point Average* 


2.81 

(0.668) 


2.66 
(0.668) 


2.855 
(0.567) 


Percent Male 


TH 


85^ 


82^ 


Percent White 


9% 


96^t 




Pre TUCE Score 


12.39" 
(3.57) 


12.37 
(3.39) 


13.0if 
(3.86) 


NuBsber of Students** 


36 


1 


205 



*The pro;)eGted university grade point average is calculated using a formula which 
includes VSAT, hBAT, and rank in high school class. Weigthts are assigned to each 
variable with rank in high school class receiving the highest weight. 
The range here is from 0.00 to U.OO. 



**Although there are more than 300 students in 223, we have selected only those 
students who have had no previous college economics. This was done so that the 
conparison is in effect between students who are experiencing their first college 
economics course. 

7Account of the difference in student characteristics will be taken in 
the regression section by including these as independent variables. However, 
no such correction will be made in the following sections. 
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In examining the data presented In Table I It would appear that the 
students In three courses do differ In some major respects. The 223 students 
have higher MSAT and VSAT scores as well as a higher projected university 
GPA. The 101 students follow with the 121 students having the lowest MSAT, 
VSAT, and projected university GPA. In terms of the other two characteristics « 
percent male and percent white, the three classes seem quite similar. The 
students do not differ significantly (at the 5% level) on the TUCE pretest. 
Finally, as stated previously, all the 101 and 121 students are freshmen — 
indeed first semester freshmen. The 223 students on the other hand ariB 
exclusively non-freshmen with about 80 percent of them being sophomores. 

Copiitive Domain 

As stated we would expect that a course such as 101 will yield less 
cognitive attainment as ordinarily measured by researchers on economic educa- 
tion than a course like 223. The basis for this expectation is straightforward; 
the time spent on economic content in lecture is less and the lab period develops 
skills which do not translate into content knowledge. Thus the question to 
be answered in this section is: what is the extent of the tradeoff? 

Before attempting to answer this question, it is important to state 
explicitly why we are willing to accept less cognitive attainment in 101 (as 
measured by TUCE). There are two reasons. First, other knowledge and skills 
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are developed as a substitute— ^philosophy of science, computer progrssning, use 
of statistics and hypothesis testing— and we feel that these are valuable in the 

Q 

context of modern economics. Second, we feel that the reduction in cognitive 
attainment will not occur evenly; that is, 101 eaphasizes the "process of 
economic analysis" and thus these students will do relatively better on problems 
%rtiich require analysis and thou^t. 

In measuring the cognitive dimension of the courses we followed standard 
procedure and utilized the TUCE Part I (macroeconomics), Form A, as both a 
pre and a posttest. Several different types of information are available from 
the TUCE and are presented below. 

The first of these is student performance on the entire 33 question exam. 
(See Table II ). 

TABLE! II 

Test of Difference of Means (t-test) on TUCE Posttest* 
Mean of Economics 101 20.89 (15.78)** 



CoB9^rison Course 


Mean 


t-value 


Significant at .05 


National Norm 


19.22 


1.70 


yes 


Economics 121 


18.31 (lU.35)** 


2.27 


yes 


Economics 223 


23.35 (17.91)** 


-2.27 


yes 



■"There was no significant difference in performance on the pre-test across the 
courses (see Table l). 



The values in parenthesis are the means on 25 questions which the instructors 
felt were covered in all three courses although the eoverage was not the same 
in breadth or depth. The results of the cooparison of means is the same for 
this comaon core as for the overall TUCE. Thus the discussion wiJJL concentrate 
on the whole TUCE. 



®The success of 101 in these areas is measured by the students' ability to 
attain behavioral objectives set prior to the course. In general the students 
do succeed in attaining the goals. 
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these results correspond to our e)qpectatlons. The 223 course is most effective 
in teaching content as neasured by TUCE. The 101 course , although yielding 
significantly less cognitive attainment than 223 > does produce significantly 
. better results than either the national norm or 121. Given that naxiaization 
of TUCJS type perfonaance is not the sole goal of 101, we fiiid that the tradeoff 
between broader types of knowledge attained in 101 aiid TUCE is not severe and 
the students actually perform quite well. In our view this tradeoff is more 
than acceptable* 

Another type of insight into cognitive attainment is available from a 
disaggregation of the TUCE posttest scores into "objective categories." These 
include recognition and understanding, siinple application, and conplex applica- 
tion. This categorisation was suggested by the authors of TUCE and results 
in three U-question divisions.^ As noted above we would expect the 101 
students to do relatively better on questions which require more coiqplex 



^It is difficult to say whether the names given the categories represent 
the types of knowledge tested. The TUOE manual defines the categories as follows: 
"Identification or 'recognition and understanding' refers to questions which eould^ 
in principle, be axkswered by reference to a text book." 

"A simple application of a principle or concept may be defined as one in which 
the studezxt demonstrates that he can utilize that principle or concept when 
its use is specified or clearly implied." 

"A coaplex application requires that the student demonstrate ability to select 
and utilise a concept or principle when its use is not specified." See Manual 
Text of Utaderstanding College Econnmics (New York: The Psychological Corporation^ 
196J3). It has been argued however that the categories represent different 
degrees of hardness as evidenced by student scores. See Darrell Lewis and Tor 
Dahl "The Text of lAiderstanding College Economics and Its Construct Validity," 
Journal of Economic Education, Spring 1971, 2, pp. 77-85. 
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thought and reaaonlng patterns, i.e. on conplex application as opposed to 
recognition questions. Table III presents the results of this dlssggregatlon 
of TUCE perfomiance. 



TABLE III 

Test of Difference of Means on Objective Categories 

TUCE Posttest 





Mean 


t-value 


Slgnificasb 
at .05 


Kecognltlon and Understanding 
EcoQ 101 - 7.39 
CoBoparison Course 
National NarnfH^ 
121 
223 


IM 
7.64 
8.16 


-0.34 
-1.12 


No 
No 


Simple Application 
Econ 101 « 6.97 
Ccoparison Course 

National Uorw/^ 

121 

223 


6.17 
5.49 
8.21 


+ 

4-2.06 
-1.89 


Yes 
Yes 


Coaplex Application 
Econ 101 » 6.53 
Ccqparlson Course 

National Norm** 

121 

223 


« 

5.03 
5.17 
6.98 


+ 

4-2.U 
•^.62 


Yes 

Mo 



*rhere vas no Bignificant difference in perfortaance on the pre-test across the 
cotirse in any of the objective categories. The *l^«/bjectlve categories 

vere as follows: recognition and understanding 5.00 (101), 5.11 (laj, 
(223); eiaple appUcatlon 4.16 (101), 3.99 (121), 4.27 ( 223), and coevlex 
application 3.2S (101), 3.27 (121), 3.63 (223). 

4MNotet Ho standard deviations are available for the national norMng data 
and thus tests cannot be made. 
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In examining these results, we find sone confirmation of our hypothesis 
that the loss in TUCE performance for the 101 students would decline as we 
moved to more complex skills. Although the 101 students attained a sig- 
nificantly lower total TUCE score than the students In 223 there was no sig- 
nificant difference on the complex application questions—a result in accord 
with our expectations. However, the major difference between the 101 and 223 
students is on the simple application questions which Is not the expected 
result. The comparison of 101 and 121 shows exactly the pattern expected, 
101 returns a lower (insignificant) dcore on recognition and understanding 
questions and in the other two categories yields higher (significant) results. 
The same pattern seems to be indicated when 101 is compared to the national 
norm. There 101 is very slightly below the national norm on recognition and 
understanding and then rises to 12% and then to 30% superiority as we move to simple 
and complex application questions. These results when added to the overall results 
in effect reduce the negative impact of the TUCE tradeoff, the loss in TUCE 

10 

performanct occurs In "skill areas" that are not stressed within the 101 course. 

The final type of insight into cognitive attainment as measured by the 
TUCE involves a disaggregation by "content categories." The TUCE developers 
have established seven content categories. An analysis of the couti^.8 on this 
basis is important for the courses differ in topical coverage and the emphasis 
given to covered topics. It is also Important because the objective categories 
of the TUCE also vary with content. 

Given the large number of categories and the difference in the number of 
items per category, it will be easier to deal with these results in terms of the 



^®0ne might add that the com^.lex application area Is the most important not 
only because It requires the more advanced skills, but also because the retention of 
this type of knowledge appears to be greatest. These results s-^ere reported by Philip 
Sanders In a paper given at the 1974 Midwest Economics Association Meetings. 
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,„c«t of ,ueatlo„s in «ch content category answered correctly on the po.t- 
«.t. Our expectation, were that the 223 etudent. would do hetter In »ost 
,r .11 catesorle. while the 101. while not perf.r.ins as well a. the 223 etudent. 
.hould perform better than the 121 students in »st or all categories. The one 
possible exception we. the content cate^ry labeled deter^nant. of econo^c 

an, ti« on growth ger». Si^ly performance across content categories should 
h, responsive to the breadth and depth of coverage given these areas in «.y 
course* 

Ih..e results. (Table IV) in general also confirm our expectations. 

TABLE IV 

content Category Results of the Tuce Posttest 
Percent Correct* 



Category 



A. Scarcity, functioning of 

ayste^* base eletDents ot supply 
and demand 

B. Macroeconomlc Accounting 

Ce Detemination of caiP (income- 
expenditure theory) 

D. Mbney. banking, and monetary policy 

B. Government fiscal policies 

F. Determinants of economic growth 

G. Policies for stabilization and growth 
*0. the pre test there were only two differences which -J-^Jf J*^^^^^^^^ 

S^tegory D 223 was Big^ific«mtly above ^^J/J'^ ^S .boveVttest. 223 
. was Significantly above 121 (t - i;^Ji;^';ies A through E and significantly 
ie significantly better than 121 on categories ^^^^^ieantly better than 

jron regi'rj rw^^h^n: ^othfr lignmcant differences appearing. 



101 


121 1 


223 


70.8% 


65. 7X 


76. 2X 


86.0 


69.8 


87.0 


A6.5 


49.3 


59.5 


64.8 


50.3 


70.9 


60.0 


47.0 


68.1 


45.8 


70.9 


63.4 


56.4 


52.2 


61*6 
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The 223 BCudents perform best In six of the seven content categoriea and are 

second best in the remaining category. The 101 students are second in five 

of the seven categories while 121 students return the highest percentage in 

the growth category, second in another, and on the other five are in third 

place. Again 101 perforins in an acceptable manner yielding its poorest 

performance in the growth area— an area not specifically covered in the course. 

It was also lowest in the determination of GNP, a result which was not expected 

but whose negative impact is reduced because of the absence of complex application 
11 

questions. 

Th«V« then are the cognitive results. As we have indicated 101 does not 
maximize TUGE performance. But the loss in performance whether measured in 
terms of total loss, loss by objective category, or by content category 
is more than conqiensated for by gains in areas which are not captured by the 
TUCE exam. It should also tae noted that these losses are only relative to a 
course (223) which has as its objective the maximization of TUCE like 
performance.^^ Perhaps even more important is the fact that performance of 
students can be influenced by the structure and coverage of materials. 

Affective Domain 

There is clearly another output of the educational process which must 
rate on a par with the purely cognitive effects. This is the impact of our 
educational endeavors on the attitudes and beliefs of the students. The 
major problem In dealing with this important area is the absence of standarized 

^^Further analysis by content category is presented in Appendix I. 

^^It should be pointed out that micro semesters of 102 and 224 return 
TUCE results which roughly correspond to the pattern noted in the macro courses. 
In addition, we found that the gap between the courses, the tradeoff, is less 
(102 Posttest of 23.19 compared with a 224 score of 23.58 with t ■ .8). This 
result was to be expected since there is much more direct concentration on 
economic content in both the lecture and the lab of 102, for philosophy of 
science and computer programming are not taught directly. , 
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inatrunents or techniques for assessing these Impacts of teaching. Nonetheless 

the attempt must be made and this section of the paper examines three possible 

measures for use in the affective area; University-wide course evaluations. 

change and final attitudes toward economics as a discipline, and changes in 

13 

« opinions toward economic issues. 

Our expectations are that there will be a tradeoff in this dimension among 
the three courses. In particular we would expect the 121 course to have more 
of an impact in the affective area than either 101 or 223. This Is because 
of the nature of the course which takes a much more topical approach to 
macroeconomics and relates the material directly to current economic questions. 
The 223 course should yield the worst results because it Is a large lecture 
class and is taken as a required course by a majority of students. 

One commonly used measure in the affective domain is the course evaluation 
form* On the particular form used at Notre Dame students are asked to rate 
the instructor and the course on a variety of items using a. scale from 1 to 4 
with 4 being the highest possible rating. Within the Economics Department the 



13 

There is some question whether all three of these elements should be ■ 
placed in the affective domain. This is especially true of the changes in 
opinions toward economic issues which some might label a* cognitive element 
rather than an affective element. See: Benjamin S. Bloom, ed. , Taxonomy of 
^ucational Objective Handbook I (Ann Arbors McKay Ck>mpany, 1956) pp. 185-189. 
Opinion hence decision changes on the part of the student regarding specific economic 
Issues reflects the attainment of an objective In the cognitive domain in a different 
sense than is measured on the TUCE examination. For example, a student may be able to 
replicate the economists' analysis of comparative advantage (e.g., TUCE, con^lex 
application or some other application question) as that relates to foreign trade* 
Howcrver, if at the end of the course the student still feels that free trade is an 
economic evil have we achieved our objective in the cognitive domain? Finally, the 
affective area might be expanded to Includes measures such as the number of stibsequent 
economics courses taken by the students and the number of students who become majors. 
In the current case such measures are not possible or useful: at Notre Dame students 
do not formally declare majors until their junior year and thus for the base year we 
would formally have no majors generated by 101 and 121 and Business students who make 
up the majority of the 223 course are much more restricted in course choice than 
Arts and Letters students. 
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scores on the Itema seldom fall below 2.5 with a typical range between 3.0 
and 3.5. The evaluation is talcen towards the end of the semester, normally 
within the last two weeks. 

The course evaluations for the three alternative approaches are reproduced 
ill Table V. In sutnnarizing these results we can begin by making three 
observations: (i) in general, 101 receives the highest rating even though 
it engendered the heaviest work load; (ii) 121 was rated close to 101 with 
major differences appearing on three items— teacher ' s ability to encourage 
independent thinking, quality of text,' and work load and (ili) 223 receives the 
lowest overall rating, and, even though this course yields superior TUCE results, 
it was rsued lowest in terms of work load. 

Examination of the patterns on the specific items or of the overall average 
indicates partial confirmation of the expectations of the tradeoff in the affective 
domain. As expected both 101 and 121 yield ratings which are superior to the 
223 course. Contrary to expectation the 101 course is more successful than 121; 
on most of the items and, thus, on the overall average 101 receives a higher rating. 

A second measure within the affective domain is student attitudes regarding 
economics as a discipline. Specifically students were given five descriptors 
of economics and were asked to rank their attitudes on a scale from 1 to 5. 
For example, one descriptor was "harcf with a 1 implying that economics was not 
very hard while a 5 means that it was very hard. The data were collected on 
a pre and post basis. Table VI presents these results for the three courses. 
Here the 101 and 223 results are quite similar, for all five descriptors the 
post survey yielded higher values than the pre survey. Thus these students 
found economics harder, more abstract, more useful, more theoretical, and 
more relevant than they had originally thought. The 121 students moved up 
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TABLE V 
University Course Bvaluation 
(l^.O Point Scale) 



1*. 



IS 

■-si 



1. Knowledge of iub^ecrt Mtter. 

2. EnthosiMB for subject. 

3. Effectiveness of class teaching or dleectlon. 

U. Ability to stlBiltte student Interest In the subject. 

5. intwest In students-e.g., availability, helpfulness, etc. 

6. fairness In dealing irlth students. 

7. Respect for student's viewpoint. 

8. Orgsnlsatlon and aansgeBent of course. 

9. What overaU rating wuld you give this teacher as 

eoB^ared with other teachers you have at Wotre DaMT 

10. Teacher's ability to lecture In a clear, Interesting 

oanner. 

11. Teacher's Ability to encourage Independent thinking. 

12. Teacher's carefulness of preparation for class. 

13. Teacher's proaptness In returning student's nork. 
lU. InteUectual level of the lecture. 

15. Quality of text. 

16. PersoiAl value of the course to you. 

17. Standards for students perforaance req^ilred for this 

course. 

18. COBDared with other courses youhave taken or are taking 

10. jj^^ ^ortt load to this course iias: 

Average 



101 1 


121 


223 


3.93 


3.91 


3.9>> 


3.96 


3.9»» 


3.86 


3.6i» 


3.56 


3.W 


3.58 


3.^*3 1 


3.1^ 


3.61 


3M 


2.89 


3.67 


3.58 


3.31 


3.70 


3.55 


3.23 


3.58 


3.65 


3.59 






^.57 


3.80 


3.70 


\ 3.71 


3.60 


1 3.25 


3.05 


3.75 


3.67 


3.7U 


3.73 


1 3.70 


3.88 


3.67 


1 3.63 


\3M 


3.1(8 


I 3.18 


3.22 


3.60 


13.52 


3.29 


3.^ 


3.31 


3.27 


3M 


3.13 


2.79 


3.67 


13.55 
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the scale on theie deterlptore bub they found economics less abstract and less 
theoretical than they had anticipated* 

TABI£ VI 

Attitudes Towards Economies as a Discipline 
(Scale of 1 to 



Descriptor 


101 


121 


223 


Pre 


Post 


Pre 


Post 


Pre 


~ Post 


Hard 


2.99 


3.61 


3.16 


3.66 


3.19 


3.58 


Abstract 


2.69 


2.80 


2.71 


2.63 


2.81 


3.00 


Useful 




k.^ 






1*.15 


•U.3»^ 


Theoretical 


3.72 


U.06 


3.6U 


3.55 


3.57 


3.62 


Relevant 




K70 


kM 




^.20 





In Inberpretatlng these results we felt that econonlcs courses » particularly 
Introductory courses, should be perceived by students as useful and relevant. 
Indeed the 121 course Is designed to stlnulate Interest In econonlcs and thus 
presuaably should yield the bluest post values on, the useful and relevant 
descriptors.^ Relative to the 223 course this result was obtained but contrary 
to our expectations the 101 course matches 121 on the useful descriptor (a 
difference of only .01) and surpasses 121 on the relevant descriptor (a difference 
of .15). StDply the tradeoff eaqpected between 101 and 121 in this area of the 
affective domsin Is not realised; Indeedi this measure as well as the previous 
measure Isply that 101 Is the most successful of the three courses. 



^It Is Ixxterestlng to note that a course can be made more useful and more 
relevant without at the ssm tine making It less abstract and less theoretical 
at least as evidenced by 101 and 223. This Is a tradeoff which many people seem 
to feel exists In any given course. Our feeling Is that usefulness and relevancy 
can be mlntalned or increased even with heavy theoretical eaiphasls If exaq^let 
of a realistic nature are continuously used In the process of theoretical de- 
velopment or If an Involved theoretical development Is subsequently applied to 
to a problem. 
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The third and final measure of the affective donuiin Is student opinion 
regarding certain economic institutions, problems, and policies. Again surveys 
were administered on a pre and post basis. On these surveys an item— say 
inflation— is presented and under it are listed a series of semantic dif- 
ferentials such as "bad-good," "inevitable-controllable," "demand pull-cost 
push," etc. Each sewntic differential represents then a continuw ranging 
from 1 to 5 with a 3 representing a middle of the road position or no opinion. 
Eight macro items were given with a total of forty-one semantic differentials. 

The interpretation of student attitudes from such instruments is far 
from clear. It can be agreed however that change in student opinion is a 
aeasure af educational impact; the educational process operates on a student 
opinion which is either poorly formulated or which is not based on knowledge 
and evidence. Consequently the educational process brings forth a change in 
that opinion. This implies that a possible measure of the success of a course 
is the size of the opinion change. 

Table VII presents these results. To suimaarize these data we can establish 
ranges of opinion change: (i) no effect-average opinion change of less than 
.100, (ii) intenaediate effect— average opinion change of between .100 and 
.500 and (iii) significant effect— average opinion change greater than .500. 
Using these ranges 223 generates the largest nuB*er of significant changes— 
eight— followed by 101 and 121 each with five significant changes. 



'15a8 stated previously, one can argue that the type of effect we are 
attempting to capture is more appropriately cognitive than affMiye. We 
havesome sympathy for this argument but include our discussion within the 
affective section because the affective area itself, as is this particular 
dimension, fairly undefined. 
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3!ABLB Vn 

Opinions of Beononica Institutftens* 
FrobleBi and Policies 
(Scale 1 to 5) 



Sndlx^ — foSXiS 
Position chMge I Position 1 Chsm 



Issue 



101 

Snaing ««. 
Position Change 



Deeision-Mikking for National 
Beononies Policy 

SiBple->CoBq;>lex 

HandoflHBsflbtftnal 

Political-Apolitical 

laq^tant to ae-IrreleTant 
to m 

iBQortant to soeiety- 
Xrrelevanb to society 

OoveroBent pending Deficit 
UndesiriAle-Desirable 
DeBDcratie-Repabliean 
Pronotes Prosperity- 
Proaotes Depression 
Increases Aggregate Denand- 
Decreases Aggregate Deaand 

QorernMnt Controls such as Wsge 

and Price Ireezes 

Ttadesirfeble-Desirable 
DeBoeratlc-Repuiblican 
Itee Eaterprise-Soeialisa 
EffeotiTS-Sneffeetive 
paeilitate Murket Operations- 
Distort Market Operations 

POTarty 

Lasiness-Laek of Opportunity 
aex^us-UtdJ^ortaot 
Murket Detendned-DiserlJiination 
Kb Jobs-Nb Skills 
In«Yitable-Can be BllMnated 
Individual Responsibility- 
Social ReponsibUity 

Inflation 
Bad-Qood 

Recession-Prosperity 
Inevltable-Controllsble 
Deaoeratic-Republlean 
Deaand PuU-Cost Push 



if.58l 
U.032 
2.1|00 1 


+.275 
-.032 
-.1»^5 


k,k52 
3.932 
1.986 


2.097 


-.26i» 


1 .905 


1.355 


-.589 


1.U32 



3.097 
3.065 


+.375 
+.176 


2.527 
2.986 


zMh 


1 '.Ikz 


1 2.3^ 


2.290 


-.571 


2.055 



2.871 
3.i*33 
3.677 
3.91»^ 

3.677 



3.355 
1.163 
2.677 

3.19>* 
3.000 

3.571 



2.633 
3.500 

3.«^52 
2.900 
3.267 



+.OUU 
+.306 
hl.08l 



+.»»6? 
-.05* 
-.209 
-.lU9 
+.056 



+.1(62 
+.250 
-.005 
-.267 
-.019 



+.07H 


h.3B5 


+.oto 1 


U.126 


-.109 


2.189 


+.2U3 


1.881 


+.09»* 


1.333 



3.027 
3.351 
3.730 
3.027 



-.068 
+,027 

-.239 

-.769 



-.128 
+.525 
-.189 



3.U33 
2.936 

2.23U 

2.21^1 



2.975 
3.509 
3. 
3. 




3.6i|if 
1M9 
2.890 

2.905 



+.320 
-.054 
+.^ 

-!23^ 



3.873 
1.503 
2.615 
2.9l>9 
3.283 



1.932 
2.973 
3.365 
3.203 
3.563 



-.257 
-.203 

1 4*. 014 

I+.05U 
+.320 



2.70U 
3.660 
3.1103 
3.019 
3.3^ 



♦.US 
+.329 
+.13*' 

-.OlH 
-.217 

+.802^ 
-.OS'^ 

-.50: 



..oa 

+.15' 
+.0^ 
•.02^ 



+.177 3.658 +.266 3.696. +.22e 



+.323 
+.002 
-.103 
-.239 
+.155 



+.257 3.676 +.22lf 3.870 +.35? 



+.5oe 

I ♦.635 
I-.I76 
.031 
I+.I79 
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m 




181 


223 


iMUt 


Ending ^ 
Position 


Change 


Ending 
Position 


Change 


Endiag 
Position 


Change 


Ugto Utaenploynent 

Deiooratie-Rapidbliean 
BcceiiioQ-I^oiiperlty 
laflatiOB-DeflAtion 
ControllablO"Fact of Life 
M-Qoodl 


3.233 
2.W 
3.167 
2.1*67 
1.935 


+.073 
+.033 
•«-.02l* 
+.238 
+.321* 


3.315 
2.321* 

3.189 
2.51*1 
1.589 


+.099 
+.256 

+.1*59 
+.365 
+.008 


3.208 
2.120 
3.1*65 
2.132 
1.767 


-.018 
•.01*2 

+.1*1*9 
-.098 

+.159 


Federal Incoae Ibx 

FrogreeilTe-Regresalye 
Good-Bad 

Deaoeratie-Republloan 
Tt— Interpriie-SoelalltB 
Too hlg)i-Voo low 
lair-ltafair 


2.097 
2.097 
2.839 
2.710 
2.806 
2.710 


-.292 
-.292 
-.078 

+.127 
+.112 
-.290 


2.1*86 
2.351 
3.027 
2.662 

2.51*1 
3.OII* 


+.222 
+.036 

+.137 
+.216 

+.089 
-.081 


1.71*2 
2.006 
2.881 
2.61*8 
2.722 
2.673 


-.833 
-.556 
-.0^5 
-.017 
+.356 
-.6M* 


laoreaiiag the Money Supply 
Htelpi Fiople-Hurti People 
InflatloB-Reeesilon 
More Gold-More Paper 
DeBoeratle-Repoblieaa 
Free BnterpriBe-Spclaliaa 


2.1*81* 
2.033 
3.677 
3.033 
2.710 


-.766 
-.189 

+p5?»* 
+.1M* 

-.033 


2.669 
2.028 

3.919 
2.959 
2.608 


-.861 

+.731 
+.821* 

-.069 
-.172 


2.296 
2.181* 
3.829 
3.006 
2.1*91 


-.838 
+.106 
+.1|^ 
•f.031 
-.297 



* One often finds that the term "opinion" is used to refer to a type of "guess" 
that a person makes when asked a question, where that guess is based on little 
. or no inforaation. After being subjected to new information consistently 
presented (e.g., the process of education) the person may be said to have 
changed his "opinion" or to have made a decision or to "have taken a specific 
position" or to have "defined his own position" on that particular question or 
' issue. In a sense a student might have "no information" on a particular issue 
at the beginning of a course and any response he would give in an opinion survey 
would be a guess or an "opinion". By the end of the semester the student may well 
have gained enough information (consistently presented) to have made a decision 
on the specific issue or question and he has therefore taken a position on that 
issue even if it is a "middle of the road" position. This is the basic problem 
of interpretation of opinion surveys that we have noted. By and large we should 
probably use the word "opinion" for the survey results at the beginning of the 
semester and "evaluation or decision" for the results at the end of the semester. 

N 

** Ending position is defined as ^^Oi where Oi represents the opinions of the ith 

n" 

student and N the number of students. Change is defined as the difference between 
ending and beginning positions. 

*** A plus sign indicates that ending position represented a higher score than initial 
position. 
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In only two instances did the significant changes occur for the same semantic 
differential in all three courses. For the intermediate range there were 
sixteen such changes for 101, and fourteen for both 121 and 223. Eight of 
these occurred simultaneously across all three courses and four are concentrated 
within the issue of poverty. 

On this basis then it appears that 101 and 223 students have a slightly 
higher propensity to change opinions than 121 students and in the case of 223 
the changes appear to be somewhat stronger. With the exception of the poverty 
issue these changes do not occur consistently, each course seems to have its own 
impact. These results are contrary to our expectations and indicate that if 
there is a tradeoff it is not between 101 and 121 (101 performs better) but 
between 101 and 223. 

Another possible measure is to examine the "no opinion" students, i.e., 

those who were noncommittal or 3 initially and remained as such on the post survey 

This case is viewed as the extreme of "no impact"; a course did not provide any 

17 

information — as perceived by the student — upon which to form an opinion. 
To summarize the data we can compare the number of semantic differentials (out of 
a total of 41) for each course for which less than 50% of the initial 3's 
remained 3's. We find that this is the case for 20 semantic differentials in 
101, 23 in 121, and 31 in 223. Thus 223 la most successful. This analysis la 
for the most part inconsistent with our expectations: inconsistent in that 223 is 
more successful than 101 and 121, but consistent in that 121 Is more successful 
than 101. 



^^Note that the ordering of effects here Is similar to the ordering on 
the cognitive results. This suggests that Indeed this measure is part and 
parcel of cognitive achievement. 

l^It is necessary to admit that no Impact for a continuing "3" position may 
be a misnomer. What rnay have in fact occurred is that the student established 
a solid foundation for a middle of the road position, changing in effect from no 
opinion to a middle of the road position. 
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While there is no agreed upon method of assessing the impact of a couu'sti 
in the affective domain and little hope of isolating course specific effects, 
the combined evidence indicates that the tradeoff expected between 101 and 121 
did not occur. To use the positive approach 101 performed much better than 
expected relative to 121. 

Regression Analysis of Cognitive Achievement 

Previously it was argued that the expected tradeoff of less cognitive 
achievement as measured by the total TUCE for 101 relative to 223 was realized. 
That conclusion, however, is suspect for as Table I indicates the 101 students had 
lower MSAT and VSAT scores than their 223 counterparts. The counter argument then 
is that the tradeoff occurred not because of a difference between courses but 
because of differences between students in the courses. In order to control 
for this possibility and other differences we can employ multiple regression 
analysis of TUCE performance. 

Before reporting the results of the regression analysis, it is useful to 
articulate our view or theory of the educational process as it pertains to 
cognitive achievement. It appears, at least to us, that three types of factors 
are operative in producing educational output although all center on or are 
processed through the student. The first may be grouped under the heading 
of human capital and measure the ability of the student. As a student enters 
an introductory course he has certain abilities. These abilities at this point 
in time are a product of his prior experience in the educational process as 
well as such things as home environment » peer group influences t and innate 
ability. In measuring the stock of human capital we are not however concerned 
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al)out sources, we merely seek an appropriate quantification of human capital 
as it bears on the students capacity to achieve cognitively in economics. 

But Just because e student brings a certain amount of human capital to 
an introductory course does not mean that he or she vill use it. This ' obvious 
statement raises the need for our second category of factors vhich we will 
label utilization rates. Specifically ve would expect that a student with a 
given amount of hunan capital would learn more the more frequently he attends 
class, the longer he studies the subject matter and so on. 

These utilization rates unfortunately capture only the quantity dimension 
of the students application of his human capital. The need to isolate or 
incorporate the quality dimension of time esqpanded on a course Introduces 
the third type of factor. For lack of a better name, we will call these 
elements efficiency variables. The argument here is straightforward— a student 
will get more out of a given hour of lecture if he thinks the lecture was 
"good"; he will get more out of a given hour reading the text if he thinks 
the text is "good. " 

We nay then specify the functional form of our view as: 

(1) CA - f (H,UR, E) 
where GA » cofipiitlve achievement' 
H ■ human capital 
, • UR • Utilisation rate and 

E a efficiency or opinions reading elements 
' eaoployed within the educational process. 

We also postulate that the relationship between each of the independent variables 

and the dependent variable is positive. 

Even in this general form, one night argue that our view of the educational 

process is a simultaneous equation system with the values of the independent 

variables in fact dependent upon each other. It seems to us that this indeed 

nay be the case. Students with lower human capital nay compensate by increasing 
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utilisation rates. Different opinions regarding elements employed within the 
educational process may be a function of the human capital of the user. Higher 
opinions of lecture or text may increase the frequency of lecture attendance 
or the number of hours spent with the text. Perhaps an even more fundamental 
question concerns the inclusion of outputs to explain outputs. The separate 
treatments of the affective and cognitive domains implies that the two areas 
are distinct and influence(i by alternative approaches. xhe above specifica- 
tion on the other hand Implies that a favorable outcome in the affective domain 
will help to achieve a favorable outcome in the cognitive domain; that is^ 
favorable opinions about economics » the instructional elements , and shifts in 
opinions about economic problems and institutions are not only desirable in 
and of themoelves but because they facilitate the attainment of cognitive 
.bjectivea (and vice versa). Carrying this logic to the extreme would imply 
absolutely no tradeoff between cognitive and affective outcom«2^a; to achieve 
one is to achieve the other. We of course do not accept this extreme argument 
but we do feel that affective outcomes may operate positivelyt up to a point* 
on cognitive output. 

Having briefly sketched our view of the learning process we must now 
operationalize the model. In terms of human capital variables three are 
employed in all regressions. They are MSAT» VSAT, and rank in high school 
class (RHSC). The latter was included on the presumption that human capital 
may not be fully captured by MSAT and VSAT. Of course the three variables 
are not perfect measures of the stock of human capital at the time the students 
beg^n the respective couraes. In the case of the 223 students who are mainly sopho- 
mores these variables measure human capital as It existed some two years previou8ly< 
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Finally, the aajor purpose of the regression analysis is to determine 
whether the differential cognitive results reported earlier could be at- 
tributed to the courses themselves. Thus we have included two dwsay variables. 
The first Is DV223 which was coded as a 1 for all 223 students and 0 for all 
* ot"..f rs. The second Is i)V121 which was coded as a 1 for all 121 students and 
zero for all others. In this way we can compare the 101 students to each of 
the other groups by examining the sign and signiflcanc i of the two dunay 
variables. 

The regression version of the model thus appears as 

CA H a + bi(MSAT)+ b2(VSAT)+ b3(RHSC)+ b^(CE20)+ b5(CB9) 

+ b6(CE15)+ b7(CEl6)+ b8(0PI) + b9(DV223)+ bio(DV12iy 

AU of the variables— excluding DV223 and J)V121— are expected to operate 
positively on cognitive achievement. The elimination of pre TUCa scores 
as Independent variables was Justified on the basis of using the gap closing 
specification of the dependent variable. Regressions were ran on the Total 
TUCB as well as the three objective categories and the results are presented in 
Table Vlll.^**" 

These results offer InsightH in many areas but the two of concern here 
are the explanatory ability of the model and the Impact' of the courses themselves. 
Wth respect to the former we can begin by examining the B? values. We find 
that the model works fairly well in explaining tot?il TOCE perfomance (R - 0.395) 
but then unevenly among the objective categories. Here the model performed best 
in the sii^le application category (R^ ■ O.U25) and worst in the recognition 
and under sttuiding category (B^ - 0.101 ). 



S^See Frank a«ry"Xs there a Ceiling Effect to the Test of Understanding 
College Economics?" Research Papers in Economic Education (Hew York: Joint 
Council on Economic Education, 1972; pp. S?-**?* 

^^ddltlonal regression analysis is contained in Appendix IX. 
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Another aipect of tht nodel's efficacy la the aign and algnlflcance of 
the coeffldenta. Of the three hucaan capital varlabXea we find that M8AT 
la the noat conalatent— algnlflcant and poaltlve In three of the four re- 
greaalona. V8AT haa one Incorrect alga but la Inalgnlf leant here and In the 
one Inatance It la algnlflcant. It haa the proper algn. RHSC la conalitently 
poaltlve but Inalgnlf leant . 

UR repreaented the alngle utlllaatlon rate varUble. It 'a perfomance 
la dlaappolntlng— Inalgnlflcant In all four regreaalon with one Incorrect alg^. 
Aa atated prevloualy the wording of the queatlon on the courae evaluation 
ioxm nay explain the poor perfomance of thla variable. 

With the four efficiency varlablea, the reaulta are alao dlaappolntlng. 
The algn of the teacher evaluation la conalatently negative and In two Inataneea 
It la algnlflcant. 22 The text evaluation haa two unexpected algna but In all 
four regreaalona la Inalgnlflcant. The courae evaluation la conalatently 
poaltlve but Inalgnlflcant. OPl operatea poaltlvely In all four regreaalona 
and la algnlflcant fn the cottplex application regreaalon. Clearly theae reaulta 
liply that the aodel needa further work and better devlcea to neaaure the 
varloua aapecta of the educational proceaa which l.t apedflea. 

Turning to the queatlon of the effect a of the courae, the algn and 
al^lflcance of the duany varlablea aupport our earlier aaaertlona. In the 
total TOCE regreaalon we find that DV223 la poaltlve while DV121 la negative 
with both varlablea algnlflcant. The 223 atudenta after controlling for the 
other Independent varlablea obtain higher total TOCE acorea than the 101 atudenta 
but the 101 atudenta are algniflcantly better than the 121 etudenta. On 
racotfiltlon and underatandlng queatlona 223 la again algniflcantly better than 



22ror a fairly complete dlecuaalon of teacher evaluatlona eee the Fall 1973 
laaue of th/f J^ mmal of Econotdc Education. 

! 

• I 
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101 but there is no difference between 101 and 121. With slBple ftpplicatione 
223 is significantly better and 121 significantly worse than 101. We had 
hoped that the 223 - 101 conparison would not be si^ificant in this category, 
on the cowplex application we do obtain our expected results—no significant 
difference between 223 and 101 and 101 is significantly better than 121. 
Sis^ply the results of the regression analysis do confliw the earlier reported 
results, again disconflrming only one of our a pr iori expectations. 

Conclusions 

In the teaching of any course the instructor nast identify his objectives, 
develop a strategy for obtaining these objectives, and determine the extent 
to which the objectives have been obtained. The same is true of a department 
which offers alternative Introductory courses. The Identification of objectives 
treated In this paper Include both co^ltlve and affective outcones for three 
different courses as well as the expected tradeoffs. The strategies for obtain- 
ing these different outcomes was developed by examining the structure of each 
course. The TUCE was used to determine the cognitive outcomes while three 
different measures were used to assess affective outccmee. In general, the 
results Indicated that our objectives In each course were being attained; 
, courses Indeed can be designed to achieve different outcomes. 

We hope that this paper wlU stlauOate others to undertake evaluations 
of their own courses. In these activities, as Indicated by oiu* own difficulties, 
spteclal attention should be given the TUCE. In particular the dual disaggrega- 
tion of the TUCE provides extensive analytical capabilities. Slniply if one is 
teaching an analytical course he should pay particular attention to the objective 
categoxy results but also be careful that the content coverage of hla course 
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pawOlels that of the TUCE, It also appears that the entire affective domain 
18 an area In which mch research needs to be done. Finally, It also appears 
that Individuals Must be careful in specifying what they believe to be the 
. educational process taking place in their courses before proceeding to 
enpirical analysis. Hopefully this paper has provided some insights in these 
latter areas which others my build upon. 
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Appendix X 
Performance by Content Categories 

As was noted in the body of the paper 223 students perform best in six of 
the seven content categories and are second in the remaining category. The 121 
students are best in one category, second in another and on the other five are 
in third place. The primary factor which seems to affect these rankings is 
the concentration which an individual course places on a given topic. For 
example, little time was spent on growth in 101 or 223 while 121 does spend 
some time on this topic. Thus the superior performance of 121 students in this 
content catei;r"»ry can be explained by concentration alone. Also most of the 
conventional macroeconomics taught in 121 is presented within the context of 
GNP or income determination and thus the better performance of these students 
with respect to 101 students here ie elso understandable. In both of these 
content categories— determination of GNP and determinants of economic growth- 
there are no complex application questions. This is one dimension of why 121 
students do better in these areas but exhibit relatively poor performance on 
complex application questions. In the same vein it is interesting to note that 
more than half of the 121 correct complex applications occurred in the area 
of policies for stabilisation and growth. Thus there 1^ a rather clear llnkege 
between the objective and content categories of TUCE. 

The data presented in Table III also seem to reflect « ceiling effect. 
For example, the maximum average attainmsnt on any content category was about 
86X ' (macroeconomlc accounting) for the 101 and 223 students. This Implies that 
in any given content area, it may be difficult to get an average attainment 
greater than 86%. This specific result may be biased upwards, however, since 
there are only three questions in this content category end it also has the 
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highest percentage of recognition and understanding questions— 66^ as compared 
to 30^, for the next highest category. If w look past this for the nowent, 
the 223 students attained 7^ on the scarcity category of six questions but 
this was the only other score which was above 7li, Indeed twelve of the renain- 
Ing scores are between 53% and 71^^ while the other six are all above Uj^. Thus 
we nay be dealing with a rather narrow range of fin^l achievement for the 
topical areas stressed In any given course. 

In light of this difficulty, the data may be anaOyaed In an alternative 
way which, hopefully, deals more directly with the goals of teaching; that 
Is, the percent Improvement In the content categories. This Is defined as a 
ratio of the difference between fosttest and pretest scores to pretest score 
and the results are presented In Table A.I, Here we find much more variation: 
In three cases we find the scores on the poattest more than double those of 
the pretest and with the exception of one case In which there was absolutely 
no improvement, the mAnloum gain Is 30^* T!heM bounds probably indicate the 
general range of effects Vor a given course. The results again indicate that 
the superior perfomance of the 223 students was the result of an across-the- 
board laprovement. In the three cases In which the 223 students are not 
superior only one appears to be a major difference— monetary— In which the 
101 students pmctlcally doubled the inprovement of the 223 students. Again 
only In the growth category do 121 students appear as superior although they 
also continue to perform better than the 101 In the QMF determination category. 

It has also been suggested both becautie of the limited number of questions 
In certain content categories and of the possible overlap of concepts between 
categories that the questions be aggregated into four categories* Hie categories 
are listed In Table A. II as well as the results first for percent correct 
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TABLE A.I.. 

Percentage Bqprovemetit in TUC£ by Content Categories 



Category 


101 




* M J 


Scarcity, functioning of economic systens, 
base element of supply ana aenana 




30.4 


— W.l 


Macroeeonomic accounting 




32.5 


71.5 


Determination of QMP (Income expendltnre 
theory) 


86.1 


128.1 


151. 


Money, banking, and monetary theory 


a26.'U 




66.7 


Government fiscal policies 


38.4 


3U.9 


79.0 


Determinants of economic growth 


0 


lf5.8 


• 37.5 


Polities for stabilization and growth 


76.8 




65.5 



on the TUCE posttest and percentage iaqprovement. With respect to percent 
correct ire find 223 superior in aU four categories but with little variation 
in these percentages between categories and courses. The lowest is kst correct 
while the bluest is 76^. In terms of percentage iap^ovement 223 is superior 
in three of the four categories with more variatloh. The range here is from 
33^ improvement to 105^ iagprovement. These results are not particularly 
different from thote using seven content categories and thus because of the 
greater detail provided by the seven categories it is to be preferred. 
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Table A. II , 
Results Using Four Content Categories 
Percent Correct on lUCE Fostteot 



Category 


101 


121 


223 


X« Scarcity 1 functioning of economics systems ^ 
base elements of supply and demand 


71^ 




76* 


2. MBtcroeconcsnic accounting and determination 
of OTP (income-expenditure theory) 






71* 


3* Money, banking^ and monetary policy and 
govemoent fiscal policies 






70* 


k* Determinants of economic growth and 

policies for stabilization and growth 






. 62* 

• 


Percent Improvement on TUCE 








Category 


JJDl 


121 


22? 


1. Scarcity, functioning of economic systems, 
base elements of supply and demand 




30* 




2« Macroeconomic accounting and determination 
of ON? (income-expenditure theory) 




66* 


X09* 


3« Money, banking, and monetary policy and 
govemaent fiscal policies 




'40* 


72* • 


Determinants of economic growth and 
' ' policies for stabilization and growth 


53?t 


33* 


57* 
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Appendix B 
Further Regression Analysis 

One question which might arise with respect to the results presented 
in Table VIII concerns the sensitivity of conclusions concerning the three 
courses to alternative definitions of the dependent variable. Two alterna- 
tive specifications of the dependent variable were tried-- absolute score 
on the TUCE posttest and percent improvenent (ratio of the difference 
bjetween post less pre to pre). In both instances the TUCE pretest was in- 
eluded as an independent variable. The results indicate that no such sensitivity 
exists— the sign and significance of the dummy variables remains as reported in 
Table VIII. As a point of Interest, R^'s are maximiaed by using percent improve- 
ment as the dependent variable and including the TUCE pretest as an independent 
variable— typical Increase on the order of .2 (an increase in explained 
variation of 20X). 

We also addressed the sex question which might be classified as a 
human capital variable. The problem here was to control simultaneously 
for sex and course. Thus five dummy variables were employed M223 (males 
in 223-1), F223 (females in 22>1), M121 (males in 121-1), F121 (females 
in 121-1), and FlOl (females in 101-1). The comparison group Is males in 
101. The results of these regressions for the total TUCE and the three 
objective categories are presented in Table B.I. Again the gap closing 
version of -the dependent variable was employed. 

Looking first at the 101 male versus female comparison we find no 
■Ignlflcance difference on the overall TUCE and two of the three objective 
categories* On recognition and understanding question 101 females are 
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■ignifleantly below their 101 nale counterparts. Both 121 males and fenales 
•re significantly below 101 males on the overall TUCE. For the 121 males 
this is also true on two of the three objective categories while only 
true in one category for the 121 females. We find no significant difference 
between 223 females and 101 males while 223 males are significantly better 
than the 101 males in two of the four regressions. In summary then there 
is some slight evidence of a sex factor in 101 and 121'~to the advantage 
of males in 101 and to the advantage of females in 121— and stronger evi- 
dence in 223-— to the advantage of males. 

As a final comment on the regression analysis there is the question of 
nulticollinearity between variables. The correlation coefficients are re- 
produced in Table B.2. for the regressions presented in Table VI. Other 
than the expected high correlation between the two dunny variables i the 
remaining correlation coefficients do not suggest any high degree of 
nullicollinearity. The variables exhibiting the highest collinearity are 
three of the efficiency variables but even here the highest correlation 
coefficient is only about .5* 



ERIC 



319V1IVAV AdOO iS39 



s 



• I • 1 • I 



t i i> ? ? r 



SI I 



&l o| Pi p 

ll' 



il *l p, . 

i 



I 



o 



bl &| o| o| 



Pip 



I i 1 1 1 1 

1 1 1 1 11 

e & & p p, . 

>l i»l .^1 d CI ?' 
f 

?\ ^\ ^'l c ?' 

II 

I i ' 

}l ll ll 1 1 S 



"I ^ j> .<> la 
8 fe k •« 



o I o . - , - 



el o I & I & 

I III 
•* 2 la 

II §1 11 1 f 
II II ill g 



^ o I o I o 

rl I I It* 



i^l j> j> P i^l ^ 

''llilllllllllgllE 



o| ^ j> j> 



M 



